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LlghtMouse is a tool for visual interaction with 
a PC (or PC -tike entertainment device) through 
y fts projected display using only a laser pointer 

LlghtMo use can wo rk in tandem with the 
common PC mous«7i 
mouse only when the user points a 
onto the projected display area 
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FIG - 10 
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FIG - 14 
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START 
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£ 



Turn display on, using the image 
from the light sensor, find display 
area based on the characteristics 
of the display or the known image 
on the display 



Turn display off, with the senso 
capture a frame of the display 
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A Turn display on, capture a frame 
/ of the display space 

T 
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Locate display by examining the 
difference between the two frames 

i 



Adjust the light sensor position and sensing 
parameters for the best viewing conditions 
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Yes 



Define the borders of the display in the image captured 
by the light sensor as continuous lines or curves 



Borders of the display as it is 
'captured by the light sensor, their 
visual characteristics, location and \ 
curvature 



Continue to pointing device 
registration (Figure 15) 
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FIG -15 
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Instruct user to register the pointing device he/she will use. The user may select a 
pointing device from a list or allow the system to view the pointing device by 
allowing the pointing device to be viewed by the light sensor. - . 
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Instruct user to point the 
pointing device to various 
points in the display. Capture 
one or more frames of the 
display space with the 
pointing device 
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Capture a frame of the display space 
without the pointing device 
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Capture a frame of the display space 
with the pointing device 
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Locate pointing device by examining 
the difference between the two 
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I - 

Adjust the light sensor position and sensing 
parameters for the best viewing conditions 
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Determine the distinguishing characteristics of the pointing device 
as opposed to the rest of the display by observing it via the light 

sensor against any image on the display or against a set of 
calibration images. Adjust the light sensor position and sensing 
parameters for optimum operation. 
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Continue to computing the mapping 
between the display space registered 
by the light sensor and the computer 
display 



Distinguishing characteristics 
of the pointing device against 
a variety of display backgrounds. 

~ — 

88 



FIG - 16 
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START 
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Turn display on, instruct the 
user to point the. pointing device 
to a convenient location of the 
display (e.g., center). Using the 
image from the light sensor, find 
the pointing device based on the 

known characteristics of the 
. pointing device. 



Instruct user to hide pointing device, with 
the light sensor, capture a frame of the 
display space 
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Instruct user to point the pointing device to a 
convenient location of the display, capture a 
frame of the display space 



Locate the pointing device by examining 
the difference between the two frames 
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Adjust the light sensor position and sensing 
parameters for best viewing conditions 
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No 



Instruct user to point with the pointing device to the borders 
and/or various locations of the display. Capture frames 
with light sensor. 
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Define the borders of the display space in the image captured 
by the light sensor as continuous lines or curves 
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Borders of the display as it is 
'captured by the light sensor, their 
visual characteristics, location and y 
curvature 



Continue to pointing device 
registration (Figure 17) 
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FIG - 17 
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Instruct user to point with the pointing device to the borders and/or various 
locations of the display. Capture frames with light sensor. 
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Adjust the light sensor position and sensing 
parameters for the best viewing conditions 
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Determine the distinguishing characteristics of the pointing device 
as opposed to the rest of the display by observing it via the light 
sensor against the background of the virtual display. Adjust the 
light sensor position and sensing parameters for optimum operation. 
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Continue to computing the mapping 
between the display space registered 
by the light sensor.and the computer 
display 
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Distinguishing characteristics 
of the pointing device against 
a variety of display backgrounds. 
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FIG - 18 
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Borders of the display as it is 
captured by the light sensor, 
their visual, characteristics, 
location and curvature 
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Distinguishing characteristics 
of the pointing device (against^ 
a variety of display 
backgrounds if real). 



Images capture with the 
pointing device pointing to 
various regions of the display.. 
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Divide the display space into the 
same number of regions as the 
those of the computer display 
using the information from the 
information from the borders of 
the display space. 



Establish correspondence between 
the display space observed by the 
light sensor and the computer display 
region by region. Make necessary 
adjustments to the boundaries of 
individual- regions as necessary. 
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Using the information from the 
position, of the pointing device 

previously registered by the user 
and the images captured when 
the user pointed the pointing 
device to the regions of the 
display as instructured by the 
mapping between the image 
space registered by the light 

sensor and the computer display. 
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Continue to system operation 
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Mapping between the display 
space and the computer 
display 
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FIG - 24A 











V* 

-4 

k 


t:-:-::-:::-:-:^:-:::::::-: 
/:::>:::x:::::>x>:::::>::x>-:ffl 

TTTrrm^fi i i rrnr— 



FIG - 24B 
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FIG - 26 
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Define region around item or icon 
that will be used to determine if a 
single click is warranted 
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highlight item. 
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Define number of frames or length 
of time that the pointer has to be in 
the region to highlight the item 



Clear stack 
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Find pointer device and position 
the computer mouse accordingly 
on the display, store mouse 
position is stack 
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Throw out the oldest mouse 
coordinate from the stack. 
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FIG -28 



System 
Operation 



Get data from sensor or 
image frame from camera 



Locate pointing device 
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Yes 



Map position of pointing 
device to point on the real 
or virtual display space 



Find the computer display 
position corresponding to., 
the pointing device position 



Position the computer's 
pointing icon (e.g., mouse 

arrow) at the computed 
computer display position 
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